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^ Abstract 
O 

In this paper the ensemble of codes formed by a serial concatenation of a repetition code with 
multiple accumulators connected through random interleavers is considered. Based on finite length weight 
psj ' enumerators for these codes, asymptotic expressions for the minimum distance and an arbitrary number 

, of accumulators larger than one are derived using the uniform interleaver approach. In accordance with 

earUer results in the literature, it is first shown that the minimum distance of repeat-accumulate codes can 
grow, at best, sublinearly with block length. Then, for repeat-accumulate-accumulate codes and rates of 
1/3 or less, it is proved that these codes exhibit asymptotically linear distance growth with block length, 
where the gap to the Gilbert- Varshamov bound can be made vanishingly small by increasing the number 
. of accumulators beyond two. In order to address larger rates, random puncturing of a low-rate mother 

' code is introduced. It is shown that in this case the resulting ensemble of repeat-accumulate-accumulate 

codes asymptotically achieves linear distance growth close to the Gilbert- Varshamov bound. This holds 
even for very high rate codes. 

Index Terms 

Multiple serial concatenation, repeat-accumulate codes, uniform interleaver, minimum distance growth 
rate coefficient, Gilbert- Varshamov bound 
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I. Introduction 

Since the invention of turbo codes, several new turbo-like coding schemes have been proposed. Among 
these are serially concatenated codes, a simple example of which is the repeat-accumulate (RA) code lUl 
consisting only of a repetition code, an interleaver, and an accumulator. Such a code has the advantage of 
very low decoding complexity compared to serially concatenated code constructions with more complex 
constituent codes. Another benefit of RA codes, compared to powerful code constructions such as LDPC 
codes, is their extremely low encoding complexity of 0(1), whereas LDPC codes have an encoding 
complexity of 0{g), where g, although much smaller than the block length [3], is greater than one. 
This makes RA codes well suited in power-limited environments; for example, for physical layer error 
correction in battery powered sensor network nodes or for spacecraft communications. 

Design guidelines for achieving an interleaver gain with double serially concatenated code constructions 
have been given in iU (and in ISl for more general parallel and serially concatenated code ensembles). In 
this paper we address multiple serially concatenated RA-type codes, where, in particular, we focus on the 
serial concatenation of an outer repetition code with two or more accumulators connected through random 
interleavers. The resulting code ensemble is then analyzed using the uniform interleaver approach 161 by 
averaging over all possible interleavers. Our work is mainly motivated by Q, [U, where a similar setup 
was considered. Whereas Q considers the interleaver gain of repeat multiple accumulate (RMA) codes 
(and thus addresses a special case of the work in Q), in HI the authors show that, for an asymptotically 
large number of accumulators, there are codes in the ensemble whose minimum distance grows linearly 
with block length and which achieve the Gilbert- Varshamov bound (GVB). They also provide numerical 
calculations of the minimum distance for different numbers of accumulators and finite block lengths, but 
they do not give results for the asymptotic minimum distance growth rate coefficient in the practically 
more relevant case of a finite (small) number of accumulators. An extension of this work was presented 
in 191, where it is shown that, for a finite number of accumulators larger than one, linear minimum 
distance growth can be obtained, but an asymptotic growth rate coefficient for the minimum distance 
was only conjectured. This was also shown in ITOl for the special case of two accumulators, but again 
an asymptotic growth rate coefficient was not given. 

Lower bounds on the average minimum distance growth for finite block lengths are given in Fig. [1] 
for different code rates R and two (RAA) and three accumulators (RAAA) along with the GVB. The 
bounds are computed such that half of the codes in each ensemble have a minimum distance of at least 
rfmin- We observe that, for rate R = 1/2, the growth rate coefficient of the RAA code is much smaller 
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than the GVB ; however, both the RAAA code and the RAA code punctured from a rate 1/3 mother code 
(R3-AAp) have a minimum distance growth rate coefficient very close to the GVB. Also, it can be seen 
that, for code rates of i? = 1/3 and R = 1/4, the minimum distance growth rate coefficient is closer 
to the GVB than in the i? = 1/2 case. For the rate R = 1/3 and i? = 1/4 RAAA code ensembles the 
minimum distance growth rate coefficients coincide with the GVB and are therefore not shown in Fig. [J 
In contrast to the results given in Fig. [T] we are interested in the asymptotic behavior of the minimum 
distance for RMA codes with block lengths tending to infinity. In particular, an asymptotically "good" 
code ensemble has the property that, for large block lengths, the minimum distance grows linearly with 
block length, which cannot be reliably determined from the finite length analysis shown in Fig. [T] 
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Fig. 1. Lower bounds on the average minimum distance growth rate coefficient for finite block lengths for RAA and RAAA 
code ensembles along with the GVB. 

In the following, we extend previous results in the literature Jll, Q, |[TOl and present an analysis that 
fully characterizes the asymptotic minimum distance behavior of RA and RMA code ensembles. The 
main result of the paper is that, for RAA codes and rates equal to 1/3 or smaller, these code ensembles 
exhibit linear distance growth with block length, where the gap to the GVB can be made arbitrarily small 
by increasing the number of accumulators beyond two. In addition, we consider random puncturing at the 
output of the last accumulator. We show that in this case the resulting ensemble of RAA codes achieves 
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linear distance growth close to the GVB for any code rate smaller than one. 

The paper is organized as follows. In Section II we consider the ensemble average weight spectrum 
of RA codes and show that the minimum distance grows as a fractional power of the block length. 
Section III addresses the asymptotic minimum distance analysis of a double serially concatenated RAA 
code. This concept is extended to multiple serial concatenation with more than two accumulators in 
Section IV. Finally, random puncturing and its effect on minimum distance is discussed in Section V, 
and some concluding remarks are given in Section VI. 

II. Ensemble average weight spectrum for repeat-accumulate codes 

In this section we briefly address the minimum distance of RA codes and show that these codes cannot 
achieve linear distance growth with block length, i.e., they are not asymptotically good code ensembles. 
Related results have already been established in [91, ifTOl . ifTTl . and |[T2l . where lower and upper bounds 
on minimum distance for more general serially concatenated codes have been derived. We restate these 
results for tutorial reasons and since we will use them in Section JIIJ where we analyze the RAA code 
ensemble. 

The RA encoder is shown in Fig. |2] The binary input sequence u has length K and Hamming weight 
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Fig. 2. Repeat-accumulate encoder. 

w, and TZ denotes the repetition code of rate R = 1/q, which leads to a codeword of weight qw and 
length = qK. The subsequent interleaver vri permutes the symbols of the codeword. We consider the 
ensemble of all interleavers by using the uniform interleaver approach ||6|, where each possible interleaver 
realization has probability 1/A^!. The permuted output sequence is applied to the recursive convolutional 
code Ai with generator polynomial g{D) = + D) (accumulator), leading to an output sequence v 
of weight d. 

We will characterize RA code ensembles by their input-output weight enumerating function (lOWEF) 
^d,w^ which is the number of codewords having weight d that result from input sequences of weight w. 
Let E{A^^^) denote the expected value of the lOWEF using the uniform interleaver approach. 
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We also define the ensemble-average weight enumerating function (WEF) E{Ad), the expected number 
of codewords of weight d, as 

K 

E{Aa)^^EiAd,J, (1) 



w=l 



and similarly the ensemble-average cumulative WEF E{Aii<s) specifying the expected number of code- 
words in the ensemble with weight not exceeding 6, 



(2) 



d=l 



For a given code, the minimum distance is defined as the smallest value of 6 for which A(i<,s is 
nonzero. 



Theorem 1. In the ensemble of RA codes with block length N 
minimum distance d^,, lower bounded by the inequality 



oo and q > 3, almost all codes have 



dmin > iV ' , 

where e is any fixed positive value. 

Proof: The conditional probability that a weight d codeword is obtained at the output of the 
accumulator for a given input weight w can be expressed as l[2l. |[6l 



VT{d\w) = < 



1, 

0, 
0, 



d-l \ fqK - d 
2 



for 1(7 = 0, d = 0, 

for tt; = 0, d> 1, 

for w > 1, d = 0, 

for w > 1, d> 1. 



(3) 



qK 
qw 

Since we are only interested in code words with nonzero weight, in the following we will only consider 
the case where w > 1. Note that from Q we obtain the constraints 



qw 



< d and 



qw 



<qK-d. 



The total number of input sequences having weight w is (;^). Then E{Ad^u]) is given as 



E{Ad, 



w 



VY{d\w). 



(4) 
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By using the fact that 



>T\{N -\)> — with ^AW=exp 



A=0 



1) 



2A 



we can show that 



1 



N 



Thus we can upper bound ([5]) as 



E{Ad,^) < 2"- 



qw 



(Pn{w). 



The cumulative WEF is given by 
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W = l (1=1 

and by using (HJ the sum over d in (|9ll can be upper bounded as 

^ E{A,,^) < ^ 



d=l 



d=l 



qw 



iPn{w) < 



f\Jqw — w— \qw 1 2J 
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qw 



(6) 



(7) 



(8) 



(9) 



(10) 



where tt; > 1, d > 1. Now choosing 6 = N i where e is any fixed positive value, we obtain 
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j^~qw(L/2 
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i/{2g)^ ^qw/e _ 



where 



q 



(11) 



(12) 



and we have employed the inequalities 

max e^-'-m2N) < ^l/{2q) ^ < ^q/e^ 

l<w<K 

Next we choose an No = No^e) such that 

" ^ q 2 
and then, for N > Nq, by combining (fTTI ). and (fT2l ) we obtain 

oo 

UI = 1 

From ([T4l ) we conclude that there exists codes in the ensemble with block length N > Nq whose 
minimum distance satisfies > N~~'^. Since 77/(1 — ry) ^ as ^ oo, the fraction of codes in 



(13) 



(14) 
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the ensemble with dj^^ < N 1 goes to zero, which proves the theorem. 
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In the case q = 2 the above bound does not hold. In particular, using ([5]) we can show that 
E{Ad=i^w=i) = 1, i-e-, on average each RA code has one codeword of weight one generated by a 
weight one input sequence. 

III. Ensemble average weight spectrum for repeat-accumulate-accumulate codes 

In this and the following section, we extend the encoder of Fig. |2] and consider a serial concatenation 
of M accumulators Ae with generator polynomials 1/(1 + D) separated by interleavers ire, 1 < £ < M, 
as shown in Fig. [3l In particular, in this section we focus on M = 2 and, based on an average weight 
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Fig. 3. Repeat multiple accumulate encoder. 



enumerator analysis, show that the resulting repeat double accumulate (RAA) code ensembles for g > 3 
are asymptotically good, i.e., as the block length N tends to infinity, almost all codes in the ensemble 
exhibit linear distance growth with block length. 

Analogous to (|3]l, the conditional probability that a weight di codeword is obtained at the output of 
the first accumulator and a weight d codeword at the output of the second accumulator for a given input 
weight w is given as 

Pr(d,di|i(j) = VT{di\w) Vv{d\di) 



di - 1 

m-1 



qK -di 
2" 



d-1 



qK - d 
"2 



qK 

qw 

where w, di, and d must satisfy the constraints 



qK 
di 



qw 



<di, 



qw 



<qK -d 



di 
2 



< d, and 



2 



<qK-d. 



After some straightforward manipulations and, recalling that N = qK, (fTSl) can be rewritten as 

N — qw \ f qw \ f di \ f N — di 
^di ic:>i/\i'ji/\i2i/v"' 12 



Pr((i, di\w) 



r qw -[ j \ r qw - 

N 
di 



di d 



(15) 



(16) 



(17) 



8 



We can now write the ensemble average conditional lOWEF as 



E{Ad,d,,u^) = {^) Piid,di\w). (18) 



An upper bound on the ensemble-average WEF is given by 

K N 



E{Ad) = y2y^E{Aa,d,,v^)<KN max max E{Ad4,,y,), (19) 

^ — ' ^ — ' l<ui<K l<(ii<A' 

w=l di=l 

and a lower bound is given by 

E{Ad) > max max E{Ad4^,w)- (20) 

l<w<K l<di<N ' ' 

In a similar way, the ensemble-average cumulative WEF E{Ad<s) can be upper bounded by 

E(Ad<s) <5KN max max max E(Add, w)- (21) 
l<d<S l<w<K l<di<N ' ' 

In the following, our goal is to show that the ensemble-average cumulative WEF tends to zero as 
— > cxo for all 5 < Pmin-A, where p^in is a lower bound on the asymptotic minimum distance growth 
rate coefficient of the ensemble. We do this by showing that N^E{Ad^di,w) —>' for all values of d, 
l<d< PminA. 

To this end, we write the weights w, di, and d as 

w = aN'', (^l=/?iV^ d = pN^, (22) 

where 0<a<b<c<l, and a, P, p are positive constants, and condition ([T6l ) must be satisfied. 
We now divide the problem into the following two cases: 

1) At least one of the weights w, di, and d is of order o{N), so at least one of the constants a, b, 
and c is less than 1. 

2) All the weights w, di, and d can be expressed as fractions of the block length A, so a = 6 = c = 1. 

Lemma 1. In the ensemble of RAA codes with block length N and q > 3, in the case where at least 
one of the weights w, di, and d is of order o(A), N'^E(Ad^di,w) ^ as N ^ oo for all values of d, 
l<d< N/2. 

The proof of Lemma [T] can be found in Appendix [A] From Lemma [T] we conclude that the contribution 
of the first case to the cumulative WEF tends to zero as the block length N tend to infinity. Thus it is 
sufficient to only consider weights that are of the same order as the block length N. 
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We now consider the second case when all the weights can be expressed as fractions of the block 
length N. Using Stirling's approximation, we have that (^) w ^N-H{d/N) ^ where H{x) = —x ln(x) — 
(1 — x) ln(l — x) is the binary entropy function. Applying this to ( fTSl ). for ^ oo we obtain 

E{Ad,d,,u,) = exp (/(a, f5,p)N + o{N)) , (23) 

where a = w/K = qw/N, [3 = di/N, and p = d/N are normalized weights, and the function 

/(a, /3, p) = ^ - Hi(3) - Hip) + H (^^^) (1 - «) 

+ aln2 + H ( {I - P) + P ln2. (24) 

Since the minimum distance of any linear code cannot be greater than N/2, we consider a fixed 
p < 1/2, and the function f{a,P,p) in (l24l ) is defined on a region Q C IR^ with boundaries 

(a) (a = 0,0 < /3 < 2p), 

(b) (0 < a < min(l,4p),/5 = a/2), 

(25) 

(c) (0 < a < min(2 -4/),4p),/3 = 2p), 

(d) (2(l-2p) <a< l,/3 = l-a/2) if p > 0.25, 

where the boundaries follow from the constraints in ([T6l ). Note that boundary (a) should be excluded 
from region Q since we exclude the case w = 0. However, for the sake of clarity, we call it a boundary 
of Q since all points (a > 0, /3 > 0) arbitrarily close to boundary (a) in Q must be included. An example 
of Q is shown for p < 0.25 in Fig. |4l Now let fp{a,/3) = f{a,P,p) be a function of a and f3 for a 



2p 



(a): (a = 0, 

< /3 < 2p) 



(c): {0<a<4p,P = 2p) 




Fig. 4. Properties of thie function f{a, (3, p): region Q foi p < 0.25. 
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fixed p < 1/2. From (l24l) it can be seen that the value of the function /p(0, 0) is zero for any given p. 
A sufficient condition for linear distance growth can be stated as follows: if fp{a, /3) is strictly negative 
for all < p < pmin and all a, < a < 2min(/3, 1 — /3), and P, < < 2min(p, 1 — p), then almost 
all codes in the ensemble have minimum distance dmin = Pmin N > p^in N. Thus, to show that pmin is a 
lower bound on the asymptotic minimum distance growth rate coefficient of the ensemble the maximum 
of fp{a, (3) for all < p < pmin must be negative in Q. 

We now address the maximization of /(a, (3, p) over a and /?, where, in principle, a maximum of the 
function fp{a, 13) can occur inside the region Q or on boundaries (b), (c), or (d). However, we will show 
below that the maximum only occurs inside the region Q. 

Lemma 2. For any given p < 1/2, the stationary points of the function fp{a,f3) satisfy the following 
system of equations: 

'P-a/2\ fl- P-a/2\ _ f a / ^^g) 



1 — a J \ 1 — a ) \ \ — a. 
( p-f3/2 \ ( l-p-p/2 \ _ ( 1-13 l3-a/2 \' 
V 1-/3 A 1-/3 y V P l-P-a/2) ■ ^^'^ 

Proof: The partial derivatives of the function /p(a, (3) are given as 

9f 1 , " \ 1, fl3-a/2\ 1 [l-f3-a/2\ , ^ 

|Z = ,„ (JL\ _ i„ ( \ + i ,„ f V i fl-P-f/^-) + ,„ 2. ,29) 



dp vi-/3y vi-/?-a/2y 2 vi-/3y 2 V 1-/? 

At the stationary points of fp{a,P), we have df/da = and df /dj3 = 0. By setting (1281 ) to zero and 

3-a/1 
l-a 

4a; (1 — x) 



letting X = ^-rr^, we obtain 



1 — a 



which is identical to (l26l ). Likewise, by setting (l29l ) to zero with y = ^^^^^ we obtain 

which leads to ([27]). ■ 
In order to find the stationary points of /p(a,/3), we must solve (l26l) and (l27l) . where p is expressed 
as a function of a and /?. We first consider (|27] ). which can be viewed as a quadratic equation in p 
with variables a and /3. It follows that, if /? > 1/2, (|27] ) has no real-valued zeros. On the other hand, if 
P < 1/2, then ( [27] ) contains two real-valued zeros: pi{a, P) < 1/2 and p2(a,/5) > 1/2. Since we only 
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consider the case p < 1/2, the solution of the quadratic equation (|27] ) in p then yields 

1 1 



p = pi{a,P) 



1 



1-/3 (5 -a/2 



13 1-/3 -a/2 



(30) 



2 2 

which relates the quantities a, (3, and p. 

Now consider (l26l ) as a quadratic equation in /3 with variable a. For a > 1/2 there are no real-valued 
zeros, but for a < 1/2 (l26l) has two real-valued zeros: /3i(a) < 1/2 and /32(a) > 1/2. Since pi{a,(3) is 
real-valued only if /3 < 1/2, the solution of the quadratic equation (l26l ) in /3 is given by 



/5 = A(«)4-ii^l/i 



1 — a 



(31) 



A pair (a,/3) maximizingj fp{a,/3) can be computed from ( [30b and ( [3T| ) for a given p. There is a 
lower limit po for which the system of equations has a solution. If p < po. (l30l) and (|3TI ) have no solution 
and /p(q;, /3) has no stationary points within Q. For p > po, ( l30l ) and ( [3T| ) yield two solutions, where 
one solution corresponds to a maximum and the other to a saddle point. Finally, there is only a single 
stationary point for p = po. Fig. [5] shows the maxima f^'^'^ inside the region Q for different values of q, 
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Fig. 5. Maxima /™'" inside the region Q for thie RAA code ensemble withi rates K = 1/3, 1/4, 1/5, 1/6. 

where we observe a zero crossing at p = pmin, < pmin < 0.5. To show that pmin is indeed the asymptotic 



'The structure of l l30t and OU suggest a slightly different but equivalent approach, where a is used as the free parameter 
and /3 and p are determined from l |30| l and l |3U . 
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minimum distance growth rate coefficient, we now address the behavior of the function fp{a, (3) at the 
boundaries of Q. 

Lemma 3. The function fp{a,(3) cannot have a maximum on boundaries (a), (b), and (d) but can have 
a maximum on boundary (c). 

Proof: We introduce the function (/9p /3(a) = /(a, /5, p) for fixed /3 and p, which is defined on a Hne 
parallel to the a-axis (see Fig. [6] for an illustration), and consider the following three cases: 

1) For < P < min(0.5, 2p), we fix the normalized weight /? and allow a to vary between and 2/3. 
In this case, (/^^^^(a) is defined on a line from boundary (a) to (b) in (l25l) . 

2) For 1/2 < /3 < 2p, we fix /3 and allow a to vary between and 2(1 — /3). In this case, the function 
ipp^f3{a) is defined on a line from boundary (a) to (d). 

3) For /3 = 2p, the normalized input weight a varies between and min(2 — 4p,Ap), i.e., ipp^pla) is 
defined on boundary (c). 

The derivative dLpp^p{a.) / da is given by (1281 ) in all three cases. Note that, at the stationary points, a and 
j3 are related by ( [28] ). independent of p: if /3 < 1/2, then a can be obtained by solving ( [3T] ). and if 
/3 > 1/2, we can find a by solving 



1 1 — a (a 

/5= n + 



2 2 V Vl-a 
The second derivative d'^ipp^p{a)/da^ is given by 



d^H^p^pia) _ 1 1 ^1 / l-2x \ 2/3-1 ^^^^ 



da^ q a{l — a) 2 \x {1 — x) J (1 — a)^ ' 

where x = ^i^/^^ and 0<x<l/2. Ata stationary point with a = ao, the corresponding /3 can be 
obtained from ( [3T] ). For such a pair (q!o,/3), we then obtain from ( [32l ) that 



da^ 



1 (1-2/3)2 



a=ao 9 "0 (1 - ao) (1 - ao) (/3 - ao/2) {I - (3 - ao/2) 
It follows that the stationary point on a constant (3 line in the (a,/3) plane corresponds to a maximum 
for each (ao,/3) pair satisfying (|3T]) . Consequently, for cases 1) and 2), the maximum of the function 
fp{a,f5) cannot be on the boundaries (a), (b), and (d). For the same reason, in case 3), the maximum of 
the line /p(a, /3)|^_2^ is located on the boundary (c). ■ 

Lemma 4. For any p < 1/2 and q >3 the function f{a,(3,p) is negative on the boundaries (a) and (b) 
of the region Q except for the point (a,/3) = (0,0), where f{0,0,p) = for any p < 1/2. 
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The proof of Lemma |4] can be found in Appendix |Bj 

A numerical analysis shows that the maximum value on boundary (c) is always less than the maximum 
inside the region Q, if it exists, or strictly negative if there is no stationary point inside Q. And since the 
function /(a, /?, p) is always negative on the boundaries (a) and (b), except for the point {a, 13) = (0, 0), 
we need not consider the values on the boundary of the region Q in ( [25] ). and we conclude that, for all 
P < Pmin> the function fp{a,(3) is negative. Thus, we obtain from Lemmas |2l [3l and |4] that p^iin is a 
lower bound on the asymptotic minimum distance growth rate coefficient of the code ensemble. 

We summarize our findings in the following theorem. 

Theorem 2. In the ensemble of RAA codes of rate -R < 1/3 and block length N ^ oo, almost all 
codes have minimum distance d,nm growing linearly with N. A lower bound on the asymptotic minimum 
distance growth rate coefficient p^^^ = d^^^/N of the ensemble can be obtained by solving the system 
of equations i[30\) and i[31h i.e., by finding the maximum of the function f{a,(3,p). 

To illustrate the behavior of the function fp{a,(3), Fig. [6] shows two examples of contour plots of 
fp{a,f3) for the RAA code ensemble with g = 3 and different values of p. For p = 0.1, displayed 




a a 

(a)p = 0.1 (b)p = 0.2 

Fig. 6. Contour plots of fp(a,P) = f{a,P,p) for a fixed p over the region Q. Maximum values can only be located along 
the dashed line within Q. 

in Fig. Oa), there is no stationary point inside the region Q. The function /p(a,/3) is decreasing 
monotonically from the origin towards boundary (c), located at the top of Fig. Sa). By contrast, for 
p = 0.2, Fig. [6tb) clearly shows a maximum inside Q. Note that the pairs (ao,/3) satisfying (|3T| ) 
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correspond to the dashed Unes in Fig. [6l which indicate the possible locations of stationary points in the 
{a, (3) plane. 

The values of pmin are listed in Table U along with the corresponding values of the GVB, the estimated 
minimum distance growth rate coefficient pmin based on a finite block length analysis analogous to the 
one presented in Fig. [TJ and the values po- We see that, especially for lower code rates, the asymptotic 
minimum distance growth rate coefficient of the RAA ensemble is close to the GVB. Also, the results 
of the finite length analysis match the asymptotic results quite well. 

TABLE I 

Asymptotic minimum distance growth rate coefficient lower bound pum, estimated minimum distance 

GROWTH rate COEFFICIENT Pmin FROM A FINITE LENGTH ANALYSIS, AND THE CORRESPONDING VALUES OF THE GVB FOR 
THE RAA ENSEMBLE WITH DIFFERENT CODE RATES. THE VALUES po DENOTE THE SMALLEST p FOR WHICH A SOLUTION 
OF THE MAXIMIZATION PROBLEM GIVEN BY AND (|27]( CAN BE OBTAINED. 



q 


pmin 


pmin 


GVB 


po 


3 


0.1323 


0.1339 


0.1740 


0.1225 


4 


0.1911 


0.1935 


0.2145 


0.1742 


5 


0.2286 


0.2312 


0.2430 


0.2075 


6 


0.2549 


0.2570 


0.2644 


0.2309 



For q = 2, there exists no such lower bound on the asymptotic minimum distance growth rate 
coefficient. In this case, for any p > the cumulative WEF of the RAA code ensemble can be lower 
bounded by 

K N pN 

pN pN ^ 

d=l (1=1 

Even though we expect the majority of codes in the ensemble to have a minimum distance that grows 
linearly with block length 191, for any fixed p > 0, there is a nonvanishing fraction of codes in the 
ensemble with minimum distance d^in < pN. Thus, for the RAA ensemble with g = 2, we cannot give 
a lower bound on the asymptotic minimum distance growth rate coefficient. 
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IV. Ensemble Average Weight Spectrum for Repeat Multiple Accumulate Codes 

We now generalize the results of the previous section to RMA codes with M > 2, i.e., with more 
than two accumulators (see Fig. [3i0. In this case, the conditional probability of the weight vector d = 
[di,d2, ■ ■ ■ , dif] for a given input weight can be written as 

f de-1 \fqK-dA 

Pr(d|^) = Fr{d^H • ^' ' ' /tV ' ^ , (34) 

\di_ij 

where Pv{di\w) is defined in ([3]). The ensemble average lOWEF is then given by 

E{Ad,J = Pr(d|ii;) = exp {f{j) N + o{N)) , (35) 

where Stirling's approximation for large has again been employed. The vector 7 contains normalized 
weights and is given by 



7 = [f3o,Pi,P2, - ■ ■ ,Pm] 



_ w di d2 _ dM 



where f3o = a and f3M — P- The function /(7) in ( [35l ) can now be written as 

/(7) = ^-Y^ Hm + Y^H (^\^§^) (1 - + ln2 (36) 

^ e=i i=i ^ ^ 1=0 



which represents a generalization of the function defined in (1241 ) to more than three normalized weight 
terms. Analogous to the derivation for the RAA case in Section |llll /(7) must now be maximized over 
a, /?!, . . . , /3m -1- Here, the same arguments for the existence of stationary points on the boundary or inside 
an M-dimensional region Qm can be made, analogous to the RAA case, where again only the maximum 
inside Qm must be considered in the maximization problem. Thus, the M + 1 tuple (a, . . . , Pm-i, p) 
maximizing 7(7) can be expressed by the following set of recursive equations: 



/3i = i-i-^Wl-f-^) and (37) 



2 2 V V 1 - a 



1 < £ < M — 1, for any a such that < a < 1/2. The derivation of this set of equations follows from 
the derivation of (l30l ) and (OTI ) in a straightforward way: (|37] ) is equivalent to (|3TI ) with f3 replaced by 
/?!, and (|38] ) is a generalization of (l30l) with p replaced by and /3 by 



Hn contrast to the M = 2 case, for Af > 2 we are able to derive a lower bound on the asymptotic minimum distance growth 
rate coefficient for the rate R = 1/2 code ensemble. 
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As an example, in Fig. |7] the values p^m for the asymptotic minimum distance growth rate coefficient 
are shown for M = 2 and M = 3 code ensembles for g = 2, 3, 4, 5, and 6 and compared to the GVB. 
The values of pmim along with the estimated values pmin obtained from a finite-length analysis and the 




Fig. 7. GVB and the corresponding asymptotic minimum distance growtli rate coefficient lower bound pmin for RAA and 
RAAA code ensembles with rates R = l/2, 1/3, 1/4, 1/5, and 1/6. 



GVB, are listed in Table |II1 We observe that the resulting asymptotic minimum distance growth rate 

TABLE II 

Asymptotic minimum distance growth rate coefficient lower bound pm™, estimated minimum distance 

GROWTH RATE COEFFICIENT Pmin FROM A FINITE LENGTH ANALYSIS, AND THE CORRESPONDING VALUES OF THE GVB FOR 

THE RAAA ENSEMBLE WITH DIFFERENT CODE RATES. 



q 


pmin 


Pmin 


GVB 


2 


0.1034 


0.1109 


0.1100 


3 


0.1731 


0.1739 


0.1740 


4 


0.2143 


0.2150 


0.2145 


5 


0.2428 


0.2400 


0.2430 


6 


0.2643 


0.2627 


0.2644 
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coefficients of the RAAA code ensemble for > 3 essentially achieve the GVB, which is consistent with 
the results obtained in HI for finite block lengths and a number of accumulators tending to infinity. 
Analogous to Theorem |2l we now state the following theorem. 

Theorem 3. In the ensemble of RMA codes with M accumulators, M > 3, of rate R <l/2 and block 
length N ^ oo, almost all codes have minimum distance d,„in growing linearly with N. A lower bound 
on the asymptotic minimum distance growth rate coefficient p^^^ = d^^ jN of the ensemble can be 
obtained by solving the system of equations (|37b and ( 1381 ). i.e., by finding the maximum of the function 
/(7)- 

V. Repeat multiple accumulate codes with random puncturing 

The rate of RMA code ensembles is determined by the rate of the outer repetition code. Thus it is 
not possible to obtain rates higher than 1/2 without puncturing. This motivates us to employ random 
puncturing at the output of the inner accumulator in connection with a lower-rate RMA mother code for 
the purpose of achieving higher rate RMA code ensembles. 

Let N' be the number of code symbols after puncturing, d' the corresponding codeword weight, and 
R' = R ■ N/N' the code rate. We also define the ratios rj = N' /N, the normalized block length, and 
p' = d' /N', the normalized output weight, after puncturing. The conditional probability of a weight-d' 
sequence after puncturing is given by the hypergeometric distribution 

'd\ f N -d 

VrN'[d'\d) 
which for large N can be expressed as 



d' j \ N' - d' 

w 

N' 



FvN'id'ld) =exp!^N 



+ o{N) } . (39) 



Considering the general case of repeat multiple accumulate codes, the ensemble average lOWEF can 
now be obtained from ( [35] ) as follows: 

E{Ad,d',n,) = Pr{d\w) FTN'{d'\d) 

= exp (F(7, p', 7])N + 0(ln N)) , 
where Pr(d|?i;) is defined in ([34] ). F(7,/9',r/) = /(7) + ip{p' , p,ri), and 

^{p', p,ii)=H p + H (^^Y^T^) ^^-P)- ^(^)- (40) 
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Following the approach used for the RAA ensemble in Section |InJ the maximization of the function 
F(7, p', 7]) must now be carried out over all elements of the vector 7, including p = (3m- Again, for the 
same reasons as in the RAA case, we consider only stationary points of F{'y,p\r]) inside the M + 1- 
dimensional region spanned by the M + 1 tuple 7. 

Note that ip{p' , p, rj) in (l40l ) does not depend on the variables a, . . . , Pm-i, since only the output 
of the inner encoder is punctured. Therefore we can still make use of (l37l l and (1381 ) for 1 < ^ < Af — 1. 
In addition, we need to compute the derivative dF/dp, which is given by 

OF ^ f p^I-p-Pm-i/2) \ , , fl-p-ii + p' 



We then solve dF/dp = for p' , which yields 



1 + C p^(l-/9-/?Af-l/2) 



We can now search for a maximum of F(7, p', -q) by using ( [37l) and (|38] ). for 1 < ^ < M — 1, and (l42l ). 

Fig. [8] considers the particularly interesting RAA case and shows the lower bound on the asymptotic 
minimum distance growth rate coefficient pmjn for different mother code rates R and punctured code rates 
R' . We observe that, compared to the unpunctured RAA code ensemble considered in Section [nil the 
asymptotic minimum distance growth rate coefficients are closer to the GVB for the punctured ensembles 
with rates R' > R. This behavior is due to the extra randomness added by puncturing the encoder output. 
We also see that the growth rate coefficients approach the GVB as the rate increases. We conjecture that 
this is due to the fact that a smaller value of rj leads to a larger random puncturing ensemble. In other 
words, a smaller N' results in a more "random-like" construction. Table |lll] gives the lower bound on 
the asymptotic minimum distance growth rate coefficient p'^^^, along with the estimated values from a 
finite-length analysis p'^^^ and the GVB, for a rate = 1/3 mother code. 

The following theorem is analogous to Theorems [2] and |3] 

Theorem 4. Consider the ensemble ofRMA codes with M accumulators, M > 2, of rate R' after random 
puncturing. If the block length N' — > 00, almost all codes in the ensemble have minimum distance d,nin 
growing linearly with N'. A lower bound on the asymptotic minimum distance growth rate coefficient 
P'miii of the ensemble can be obtained by solving the system of equations (1371 ). (I38I ). and (1421 ). i.e., by 
finding the maximum of the function F{'y,p',r]). 

VI. Concluding remarks 

We have shown that RAA code ensembles for code rates equal to 1/3 or smaller are asymptotically 
good in the sense that their average minimum distance grows linearly with block length as ^ 00. 
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Fig. 8. GVB and the corresponding normalized asymptotic minimum distance lower bound /3[„i„ for thie randomly punctured 
RAA code ensemble with mother code rates R — 1/q — 1/3, 1/4, 1/5. 

TABLE III 

Asymptotic minimum distance growth rate coefficient lower bound p^.n, estimated minimum distance 
from a finite length analysis, and the corresponding values of the gvb for the punctured raa 

ensemble with DIFFERENT CODE RATES EMPLOYING A MOTHER CODE OF RATE 1/3. 



R 


Pmin 


-/ 

Pmin 


GVB 


0.4 


0.1242 


0.1256 


0.1461 


0.5 


0.1036 


0.1039 


0.1100 


0.6 


0.0771 


0.0782 


0.0794 


0.7 


0.0522 


0.0530 


0.0532 


0.8 


0.0306 


0.0314 


0.0311 


0.9 


0.0125 


0.0133 


0.0130 



Moreover, we have shown that the asymptotic growth rate coefficients approach the GVB for small code 
rates. This extends the results of HI, where linear distance growth is only shown for an infinite number 
of accumulators. These new results also extend those in 191 and lITOl . where linear distance growth for 
the RAA ensemble is shown, but a growth rate coefficient is either not given or only conjectured. Similar 
results are also obtained for RMA code ensembles with Af > 3 and code rates equal to 1/2 or smaller. 
Further, by introducing random puncturing at the output of the inner accumulator, we demonstrate that 
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the resulting high rate RMA ensembles exhibit linear distance growth, where the asymptotic growth rate 
coefficient is close to the GVB if the mother code rate is sufficiently low. 

Despite the fact that the RMA code ensembles considered in this paper are asymptotically good, the 
convergence behavior of these codes may not be sufficient to provide an iterative decoding threshold close 
to capacity, as can be seen from the simulation results presented in [Sj]. However, the results obtained may 
be useful in constructing similar code ensembles based on simple component codes with low encoding 
complexity, asymptotically linear distance growth, and good convergence behavior. In particular, for the 
interesting class of double serially concatenated codes, the RAA ensemble can be used as a starting 
point to design asymptotically good code constructions by replacing one or more of the constituent 
encoders with small memory convolutional codes, whose code polynomials can be chosen to improve 
iterative decoding convergence behavior. Some initial experimental results in this direction for different 
component encoders are presented in |fT3l . lfT4l . Another approach is to consider hybrid concatenated 
coding schemes, where parts of the encoder are structurally equivalent to RAA encoders. Initial results for 
low code rates have shown that these schemes have improved threshold behavior compared to RAA codes, 
while still providing asymptotically linear distance growth, albeit with a smaller growth rate coefficient 

nsi. 

Appendix 

A. Proof of Lemma 1 

We consider the following five partial cases: 

2 

l--<a = h<c<l (43) 

q 

l--<a = h<c<l (44) 

0<a<6<c<l (45) 
0<a<6<c=l (46) 
< a < 6 = c =1. (47) 

From Theorem [U it follows that di > (3 N^~'^/'^ for almost all codes in the ensemble and thus we do not 
need to consider the case of 6 < 1 — |. 

In addition to (|7]), we will use the inequality |[T6l 



l^N - I) ^ \ \N J J - \ l J - \ 2ttI{N - I) ^ V 
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or the equivalent expression 



exp(i?(^) N + oiN) 



(49) 



to bound binomial coefficients, 
(a) Cases (|43]l and (l44l) : 

Using ©, ([221), and (l48]l, we can rewrite E{Ad,^ 



) Pr((ii|i(;) as 



where 



N 



qaN" 



exp ( ( 1^ ) /?iV» 



/ 2iV 



/ N 



qaN'' 



cn{w) 



exp[o(iV" IniV)] 



cn{w), (50) 



(51) 



is a second order term. For simplicity we assume [|] = [|J = |, which is vaUd for any even 
integer x and approximately valid for large odd x. Then we have from (l50l) that 



exp 



l-^]aN''\nN + o{N'' In iV) 



(52) 



Since E{Aci^^w) ^ as ^ oo, it follows that \\mK[^ooN^E{Ad^di,w) = independently of d 
for almost all codes in the ensemble, 
(b) Case (|45]l: 

To make the expressions more compact, we will omit second order terms from now on. Again, using 
(|7]l, dllll, and (|48]l, we can write 



N 



qaN" 



-qaN" 



(53) 



: exp 



((1 - a) (l - I) a + (6 - a)y) A^^lniV + 0(7^^ IniV) 



and 



:exp( -(l-6)-A^''lniV + o(iV^lnA^) 



-fSN'' 



(54) 



Since 6 > a, it follows from (1531 ) and (1541 ) that 



E {Ad,d,, rn) = exp {-{l-b)^N'' In N + oiNHnN) 



(55) 



and Af3S(^d,di,t«) ^ as ^ oo. 
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(c) Case (Hill: 

In this case E{Adi^w) is still given by (1531 ). but 



f2pN 



'-N" 



2(l-p)iV \ 



:exp ( f iVMn(4p(l - p)) + o(iV^) ) . 



Pr(d|(ii) 



Since 6 > a, it follows from ( [53] ) and ( [56l ) that 

= exp (^^iVMn(4p(l - p)) + o(iV^)) 

and N^E{Ad,d,,w) ^ as ^ oo for p < 1/2. 
(d) Case (|47]l: 

In this case ,„) is still given by ( [53] ). but 



(56) 



(57) 



Pr(d|di 



di \fN-di 



■■ exp 



d 

{1-P)-H{p) ]N + oiN) 



(58) 



The function 



Fp{f3) = (3ln2 + H 



1-/3 



(59) 



is strictly negative for all /? < 1/2 and /? < 2p, which can be shown as follows. The derivative of 



5F_1, fp-p/2\ , 1^ fl -p-p/2 

l-P 



In + - In 

dp 2 \ ) 2 



+ ln2 



1 



(60) 



In (4x(l - x)) 



is negative if x = ^^j^^ 7^ 1/2 and equals zero if x = 1/2. Fp{P) is thus negative for all p < 1/2 
and all < /3 < 2p. 
From ([53]) and ([58]) follows that 

>-/3/2^ 



= exp 



Pln2 + H 



1-/3 



(l-/3)-i/(p) iV + o(Af) 



(61) 



and N^E{Ad,d,,ui) ^ as ^ oo for all p < 1/2. 
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B. Proof of Lemma 4 

(a) On the boundary (a = 0, < /3 < 2p), we have 

df 



df3 



a=0 



1 



ln(4x;3(l-2;/3)) <0 



(62) 



where = ^j^. Since /(a, /3, p)|^^q^^^q 
all /3 > and p < 1/2. 



0, we obtain f{a, P,p) < on the boundary (a) for 



(b) On the boundary (0 < a < min{l,Ap), (3 = a/2) the total derivative with respect to a is given as 



da 



/3=q/2 



M 

da 



l3=a/2 

q 



9/3 



/3=q/2 



da 



a \ ^ 1 / a 2 

+ In 2 + - n '-— 

I- a) 2 \l-a 2 



Note that the inequality 



1 

+ 2 



iln 



2 V 1-/5 y 2 V 1-/? 



+ ln2 



1-ay 2 \l-a/2 



+ ln2 



(63) 



(64) 



holds for g > 3 and < a < 1. The last term on the right hand side of (1631 ) is equivalent to (1621) . By 
inserting (l62l ) and (l64l ) into (l63l) we obtain (i//c?a | ^_Q,y2 ^ ^- Therefore, we conclude that /(a, /5, p) 
is negative along the boundary (0 < a < min(l,4p),/3 = a/2) for all < p < 0.5. 
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